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not  every  response  that  appears  to  contradict  an  established 
fact  or  an  accepted  rule  is  a  judgmental  error.  The 
contradiction  could  also  arise  from  the  subject's 
misunderstanding  of  the  question,  or  from  the  investigator's 
misinterpretation  of  the  answer.  The  description  of  a 
particular  response  as  an  error  of  judgment  therefore 
involves  assumptions  about  the  communication  betueen  the 
experimenter  and  the  subject.  (Me  shall  return  to  this 
issue  later  in  the  paper.)  The  student  of  judgment  should 
avoid  overly  strict  interpretations.  which  treat  reasonable 
answers  as  errors,  as  well  as  overly  charitable 
interpretations,  which  attempt  to  rationalize  every 
response . 


Although  errors  of  judgment  are  but  a  method  by  which 
some  cognitive  processes  are  studied,  the  method  has  become 
a  significant  part  of  the  message.  The  accumulation  of 
demonstrations  in  which  intelligent  people  violate 
elementary  rules  of  logic  or  statistics  has  raised  doubts 
about  the  descriptive  adequacy  of  rational  models  of 
judgment  and  decision  making.  In  the  two  decades  following 
World  War  II,  several  descriptive  treatments  of  actual 
behavior  were  based  on  normative  models*  subjective 
expected  utility  theory  in  analyses  of  risky  choice,  the 
Bayesian  calculus  in  investigations  of  changes  of  belief, 
and  signal-detection  theory  in  studies  of  psychophysical 
tasks.  The  theoretical  analyses  of  these  situations,  and  to 
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a  much  lesser  degree  the  experimental  results*  suggested  an 
image  of  people  as  efficient*  nearly  optimal  decision¬ 
makers.  On  this  background*  observations  of  elementary 
violations  of  logical  or  statistical  reasoning  appeared 
surprising*  and  the  surprise  may  have  encouraged  a  view  of 
the  human  intellect  that  some  authors  have  criticized  as 
unfairly  negative  (Eduards,  1975;  Cohen,  1979,  1981;  Einhorn 
G  Hogarth*  1981 ) . 


There  are  three  related  reasons  for  the  focus  on 
systematic  errors  and  inferential  biases  in  the  study  of 
reasoning.  First*  they  expose  some  of  our  intellectual 
limitations  and  suggest  ways  of  improving  the  quality  of  our 
thinking.  Second,  errors  and  biases  often  reveal  the 
psychological  processes  and  the  heuristic  procedures  that 


govern  judgment  and  inference. 


Third,  mistakes  and 


fallacies  help  the  mapping  of  human  intuitions  by  indicating 
which  principles  of  statistics  or  logic  are  non-intuitive  or 
counter-intuitive. 


The  terms  "intuition"  and  "intuitive"  are  used  in  three 
different  senses.  First,  a  judgment  is  called  intuitive  if 
it  is  reached  by  an  informal  and  unstructured  mode  of 
reasoning*  without  the  use  of  analytic  methods  or  deliberate 
calculation.  For  example*  most  psychologists  follow  an 
intuitive  procedure  in  deciding  the  size  of  their  samples 
but  adopt  analytic  procedures  to  test  the  statistical 
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significance  of  their  results.  Second,  a  formal  rule  or  a 
fact  of  nature  is  called  intuitive  if  it  is  compatible  with 
our  lay  model  of  the  world.  Thus,  it  fa:  intuitively  obvious 
that  the  probability  of  winning  a  lottery  prize  decreases 
with  the  number  of  tickets,  but  it  is  counter-intuitive  that 
there  is  a  better  than  even  chance  that  a  group  of  23  people 
will  include  a  pair  of  individuals  with  the  same  birthday. 
Third.  a  rule  or  a  procedure  is  said  to  be  part  of  our 
repertoire  of  intuitions  when  we  apply  the  rule  or  follow 
the  procedure  in  our  normal  conduct.  The  rules  of  grammar, 
for  example,  are  part  of  the  intuitions  of  a  native  speaker, 
and  some  (though  not  all)  of  the  rules  of  plane  geometry  are 
incorporated  into  our  spatial  reasoning. 

The  present  paper  addresses  several  methodological  and 
conceptual  problems  that  arise  in  attempts  to  map  people's 
intuitions  about  chance  and  uncertainty.  Ue  begin  by 
discussing  different  tests  of  statistical  intuitions,  we 
then  turn  to  a  critique  of  the  question-answering  paradigm 
in  judgment  research,  and  ue  conclude  with  a  discussion  of 
the  non-intuitive  character  of  some  statistical  laws. 
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TESTS  OF  STATISTICAL  INTUITIONS 

Errors  and  biases  in  judgment  under  uncertainty  are  the 
major  source  of  data  for  the  mapping  of  the  boundaries  of 
people's  statistical  intuitions.  In  this  context  it  is 
instructive  to  distinguish  between  errors  of  application  and 
errors  of  comprehension.  A  failure  in  a  particular  problem 
is  called  an  error  of  application  if  there  is  evidence  that 
people  know  and  accept  a  rule  that  they  did  not  apply.  A 
failure  is  called  an  error  of  comprehension  if  people  do  not 
recognize  the  validity  of  the  rule  that  they  violated. 

An  error  of  application  is  most  convincingly 
demonstrated  when  a  person,  spontaneously  or  with  minimal 
prompting,  clutches  his  head  and  exclaims*  "Hou  could  I 
have  missed  that?".  Although  many  readers  will  recognize 
this  experience,  such  displays  of  emotion  cannot  be  counted 
on,  and  other  procedures  must  be  developed  to  demonstrate 
that  people  understand  a  rule  that  they  have  violated. 

The  understanding  of  a  rule  can  be  tested  by  (1) 
eliciting  from  subjects,  or  (2)  asking  them  to  endorse,  a 
statement  of  (1)a  general  rule,  or  (2)  an  argument  for  or 
against  a  particular  conclusion.  The  combination  of  these 
features  yields  four  procedures,  which  we  now  illustrate  and 
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discuss . 

Ue  begin  with  an  informal  example  in  uhich  understanding 
of  a  rule  is  confirmed  by  the  acceptance  or  endorsement  of 
an  argument.  One  of  us  has  presented  the  following  question 
to  many  squash  players. 


"As  you  know,  a  game  of  squash  can  be  played  either  to 
9  or  to  IS  points.  Holding  all  other  rules  of  the  game 
constant,  if  A  is  a  better  player  than  B,  uhich  scoring 
system  will  give  A  a  better  chance  of  winning?". 

Although  all  our  informants  had  some  knowledge  of 
statistics,  most  of  them  said  that  the  scoring  system  should 
not  make  any  difference.  They  were  then  asked  to  consider 
the  argument  that  the  better  player  should  prefer  the  longer 
game,  because  an  atypical  outcome  is  less  likely  to  occur  in 
a  large  sample  than  in  a  small  one.  With  very  feu 
exceptions,  the  respondents  immediately  accepted  the 
argument,  and  admitted  that  their  initial  response  had  been 
a  mistake.  Evidently,  our  informants  had  some  appreciation 
of  the  effect  of  sample  size  on  sampling  errors,  but  they 
failed  to  code  the  length  of  a  squash  game  as  an  instance  of 
sample  size.  The  fact  that  the  correct  conclusion  becomes 


compelling  as  soon  as  this  connection  is  made  indicates  that 
the  initial  response  was  an  error  of  application,  not  of 
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comprehension. 

A  more  systematic  attempt  to  diagnose  the  nature  of  an 
error  uas  made  in  a  study  of  a  phenomenon  labelled  the 
conjunction  effect  (Tversky  £  Xahnenan.  1982).  Perhaps  the 
most  elementary  principle  of  probability  theory  is  the 
conjunction  rule.  which  states  that  the  probability  of  a 
conjunction  A£B  cannot  exceed  either  the  probability  of  A  or 
the  probability  of  B.  As  the  following  example  shows, 
however.  it  is  possible  to  construct  tests  in  which  most 
judges  -  even  highly  sophisticated  ones  -  state  that  a 
conjunction  of  events  is  more  probable  than  one  of  its 
components . 

To  induce  the  conjunction  effect.  we  presented  subjects 
with  personality  sketches  of  the  type  illustrated  below: 


"Linda  is  31  years  old.  single.  outspoken,  and  very 
bright.  She  majored  in  philosophy.  As  a  student,  she 
was  deeply  concerned  with  issues  of  discrimination  and 
social  justice.  and  also  participated  in  anti-nuclear 
demonstrations . " 

In  one  version  of  the  problem.  respondents  were  asked 
which  of  two  statements  about  Linda  was  more  probable:  A. 


Linda  is  a  bank  teller;  B. 


Linda  is  a  bank  teller  who  is 
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active  in  the  feminist  movement.  In  a  large  sample  of 
statistically  naive  undergraduates ,  8654  judged  the  second 
statement  to  be  more  probable.  In  a  sample  of  psychology 
graduate  students,  only  502  committed  this  error.  Houever. 
the  difference  between  statistically  naive  and  sophisticated 
respondents  vanished  when  the  two  critical  items  were 
embedded  in  a  list  of  eight  comparable  statements  about 
Linda.  Over  80%  of  both  groups  exhibited  the  conjunction 
effect.  Similar  results  were  obtained  in  a  betueen-sub ject 
design.  in  which  the  critical  categories  were  compared 
indirectly  (Tversky  £  Kahneman  (1982). 

Tests  of  rule-endorsement  and  argument-endorsement  were 
used  in  an  effort  to  determine  whether  people  understand  and 
accept  the  conjunction  rule.  First,  we  presented  a  group  of 
statistically  naive  college  students  with  several  rule-like 
statements.  which  they  were  to  classify  as  true  or  false. 
The  statement:  "The  probability  of  X  is  always  greater  than 
the  probability  of  X  and  Y"  was  endorsed  by  81%  of 
respondents.  For  comparison,  only  6%  endorsed  "If  A  is  more 
probable  than  B.  then  they  cannot  both  occur".  These 
results  indicate  some  understanding  of  the  conjunction  rule, 
although  the  endorsement  is  not  unanimous.  perhaps  because 
of  the  abstract  and  unfamiliar  formulation. 

An  argument-endorsement  procedure  was  also  employed.  in 
which  respondents  were  given  the  description  of  Linda, 


followed  by  statements  A  and  B  above*  and  uere  asked  to 
check  which  of  the  following  arguments  they  considered 
correct : 

(i)  A  is  more  probable  than  B  because  the  probability 
that  Linda  is  both  a  bank  teller  and  an  active  feminist 
must  be  smaller  than  the  probability  that  she  is  a  bank 
teller. 

(ii)  B  is  more  probable  than  A  because  Linda  resembles 
a  bank  teller  who  is  active  in  the  feminist  movement 
more  than  she  resembles  a  bank  teller. 

Argument  (i)  favoring  the  conjunction  rule  was  endorsed 
by  8354  of  the  psychology  graduate  students,  but  only  by  4354 
of  the  statistically  naive  undergraduates.  Extensive 
discussions  with  respondents  confirmed  this  pattern. 
Statistically  sophisticated  respondents  immediately 
recognized  the  validity  of  the  conjunction  rule.  Naive 
respondents,  on  the  other  hand,  were  much  less  impressed  by 
normative  arguments,  and  many  remained  committed  to  their 
initial  responses  that  uere  inconsistent  with  the 
conjunction  rule. 

Much  to  our  surprise,  naive  subjects  did  not  have  a 
solid  grasp  of  the  conjunction  rule;  they  tended  to  endorse 
it  in  the  abstract  but  not  when  it  conflicted  with  a  strong 
impression  of  representativeness.  On  the  other  hand, 
statistically  trained  subjects  recognized  the  validity  of 
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the  rule.  and  were  able  to  apply  it  in  an  especially 
transparent  problem.  Statistical  sophistication.  houever. 
did  not  prevent  the  conjunction  effect  in  less  transparent 
version*  of  the  same  problem.  In  terms  of  the  present 
treatment.  the  conjunction  effect  appears  to  be  an  error  of 
application.  at  least  for  the  more  sophisticated  subjects, 
for  further  discussion  of  this  issue  see  Tversky  and 
Kahneman  (1982). 

In  n  attempt  to  describe  the  statistical  intuitions  of 
people  at  various  levels  of  sophistication,  Nisbett,  Krantz, 
Jepson  £  Fong  (1982)  used  an  elicitation  procedure,  in  which 
respondents  were  required  to  evaluate  and  justify  certain 
conclusions  and  inferences  attributed  to  characters  in  brief 
stories.  The  investigators  observed  large  individual 
differences  in  the  comprehension  of  basic  statistical 
principles,  which  were  highly  correlated  with  the  level  of 
statistical  training.  Naturally,  statistical  intuitions 
vary  with  intelligence,  experience,  and  education.  As  in 
other  forms  of  knowledge,  what  is  intuitive  for  the  expert 
is  often  non-intui ti ve  for  the  novice  (see  e.g.,  Larkin, 
McDermott,  Simon  £  Simon,  1980).  Nevertheless,  some 
statistical  results  (e.g.,  the  matching  birthdays  or  the 
change  of  lead  in  a  coin-tossing  game)  remain  counter¬ 
intuitive  even  for  students  of  probability  theory  (Feller, 
1968,  p.  85).  Furthermore,  there  is  some  evidence  that 
errors  (e.g.,  the  gambler's  fallacy)  which  are  commonly 


committed  by  naive  respondents  can  also  be  elicited  iron 
statistically  sophisticated  ones,  with  problems  of  greater 
subtlety  (Tversky  £  Kahneman,  1971). 

The  elicitation  method  uas  also  used  by  Uason  and  Evans 
(1975;  Evans  and  Uason,  1976)  in  studies  oi  logical 
intuitions  in  the  uell  known  four-card  problem  (Uason, 
1966).  In  the  standard  version  of  this  problem,  the 
experimenter  displays  four  cards  showing  A,  T,  4  and  7,  and 
asks  subjects  to  identify  the  cards  that  should  be  turned 
over  to  test  the  rule  "if  a  card  has  a  vowel  on  one  side,  it 
has  an  even  number  on  the  other".  The  correct  response  is 
that  the  cards  showing  A  and  7  should  be  examined,  because 
the  observation  of  an  odd  number  on  the  first  card  or  a 
vowel  on  the  second  would  refute  the  rule.  In  a  striking 
failure  of  logical  reasoning,  most  subjects  elect  to  look  at 
the  hidden  side  of  the  cards  showing  A  and  4.  Uason  and 
Evans  investigated  different  versions  of  this  problem,  and 
required  their  subjects  to  give  reasons  or  arguments  for 
their  decisions  of  whether  or  not  to  look  at  the  hidden  side 
of  each  of  the  four  cards.  The  investigators  concluded  that 
the  arguments  by  which  subjects  justified  their  responses 
were  mere  rationalizations ,  rather  than  statements  of  rules 
that  actually  guided  their  decisions. 

Other  evidence  for  people's  inadequate  understanding  of 
the  rules  of  verification  uas  reported  by  Uason  (1969)  and 
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by  Mason  and  Johnson-Lai rd  (1970).  In  order  to  provide 
"therapy"*  these  investigators  confronted  subjects  with  the 
consequences  of  their  judgments  and  called  the  subjects' 
attention  to  their  inconsistent  answers.  This  procedure  had 
little  effect  on  subsequent  performance  in  the  same  task. 
Taken  together*  the  results  suggest  that  people's 
difficulties  in  the  verification  task  reflect  a  failure  of 
comprehension,  not  of  application. 

The  examples  that  ue  have  considered  so  far  involved  the 
endorsement  of  rules  or  arguments  and  the  elicitation  of 
arguments  to  justify  a  particular  response.  Ue  have  not 
discussed  the  procedure  of  asking  respondents  to  state  the 
relevant  rule  because  such  a  test  is  often  unreasonably 
demanding:  ue  may  want  to  credit  people  with  understanding 
of  rules  that  they  cannot  articulate  properly. 

The  preferred  procedures  for  establishing  an  error  of 
application  require  a  comparison  of  people's  responses  to  a 
particular  case  with  their  judgment  about  a  relevant  rule  or 
argument  (McClelland  C  Rohrbaugh*  1978;  Slovic  C  Tversky* 
1974).  It  is  also  possible  to  confirm  an  error  of 
application  in  other  research  designs.  For  example*  Hamill* 
Wilson  and  Nisbett  (1980)  showed  subjects  an  videotaped 
interview  allegedly  conducted  with  a  prison  guard.  Half  the 
subjects  were  told  that  the  opinions  of  the  guard  (very 
humane  or  quite  brutal)  were  typical  of  prison  personnel* 
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while  the  other  subjects  uere  told  that  the  guard's 
attitudes  uere  atypical,  and  that  he  uas  either  much  more  or 
much  less  humane  than  most  oi  his  colleagues.  The  subjects 
then  estimated  the  typical  attitudes  of  prison  personnel  on 
a  variety  ox  issues.  The  surprising  result  of  the  study  uas 
that  the  opinions  expressed  by  an  atypical  guard  had  almost 
as  much  impact  on  generalizations  as  did  opinions  attributed 
to  a  typical  member  of  the  group.  Something  is  obviously 
wrong  in  this  pattern  of  judgments.  although  is  is 
impossible  to  describe  any  particular  judgment  as  erroneous, 
and  unlikely  that  many  subjects  would  realize  that  they  had 
not  been  influenced  by  the  information  about  the  guard's 
typicality  (Nisbett  £  Wilson.  1977).  In  this  case  and  in 
other  betueen-sub ject  studies.  it  appears  reasonable  to 
conclude  that  an  error  of  application  uas  made  if  the 
betueen-group  comparison  yields  a  result  that  most  people 
would  consider  untenable. 

Ue  have  defined  an  error  of  application  as  a  response 
that  violates  a  valid  rule  that  the  individual  understands 
and  accepts.  However,  it  is  often  difficult  to  determine 
the  nature  of  an  error,  because  different  tests  of  the 
understanding  and  acceptance  of  a  rule  may  yield  different 
results.  Furthermore,  the  same  rule  may  be  violated  in  one 
problem  context  and  not  in  another.  The  verification  task 
provides  a  striking  example:  subjects  uho  did  not  correctly 
verify  the  rule  "if  a  card  has  a  vowel  on  one  side,  it  has 
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an  «v«n  number  on  the  other"  had  no  difficulty  in  verifying 
a  formally  equivalent  rule:  "if  a  letter  is  sealed  it  has  a 
five  cent  stamp"  (see  Uason  £  Shapiro*  1971:  Johnson-Lai rd , 
Legrenzi  £  Sonino-Leg r enzi »  1972:  Johnson-Laird  £  Uason* 
1977) . 

These  results  illustrate  a  typical  pattern  in  the  study 
of  reasoning.  It  appears  that  people  do  not  possess  a  valid 
general  rule  for  the  verification  of  if-statements .  or  else 
they  uould  solve  the  card  problem.  On  the  other  hand,  they 
are  not  blind  to  the  correct  rule  or  else  they  uould  also 
fail  the  stamp  problem.  The  statement  that  people  do  not 
possess  the  correct  intuition  is,  strictly  speaking*  correct 
—  if  possession  of  a  rule  is  taken  to  mean  that  it  is 
always  followed.  On  the  other  hand*  this  statement  may  be 
misleading  since  it  could  suggest  a  more  general  deficit 
than  is  in  fact  observed. 

Several  conclusions  of  early  studies  of 
representativeness  appear  to  have  a  similar  status.  It  has 
been  demonstrated  that  many  adults  do  not  have  generally 
valid  intuitions  corresponding  to  the  lau  of  large  numbers* 
the  role  of  base  rates  in  Bayesian  inference*  or  the 
principles  of  regressive  prediction.  But  it  is  simply  not 
the  case  that  every  problem  to  which  these  rules  are 
relevant  will  be  answered  incorrectly*  or  that  the  rules 
cannot  appear  compelling  in  particular  contexts. 
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The  properties  that  make  formally  equivalent  problems 
easy  or  hard  to  solve  appear  to  be  related  to  the  mental 
models.  or  schemas,  which  the  problems  evoke  (Rumelhart. 
1979).  For  example,  it  seems  easier  to  see  the  relevance  of 
"not-q"  to  the  implication  "p  implies  qn  in  a  quality- 
control  schema  (did  they  forget  to  stamp  the  sealed  letter?) 
than  in  a  confirmation  schema  (does  the  negation  of  the 
conclusion  imply  the  negation  of  the  hypothesis?).  It 
appears  that  the  actual  reasoning  process  is  schema-bound  or 
content-bound  so  that  different  operations  or  inferential 
rules  are  available  in  different  contexts  (Hayes  £  Simon. 
1978).  Consequently,  human  reasoning  cannot  be  adequately 
described  in  terms  of  content-independent  formal  rules. 

The  problem  of  mapping  statistical  or  logical  intuitions 
is  further  complicated  by  the  possibility  of  reaching  highly 
unexpected  conclusions  by  a  series  of  highly  intuitive 
steps.  It  was  this  method  that  Socrates  employed  with  great 
success  to  convince  his  naive  disciples  that  they  had  always 
known  truths,  which  he  was  only  then  making  them  discover. 
Should  any  conclusions  that  can  be  reached  by  a  series  of 
intuitive  steps  be  considered  intuitive?  Braine  (1978) 
discussed  this  question  in  the  context  of  deductive 
reasoning,  and  he  proposed  immediacy  as  a  test:  A  statement 
is  intuitive  only  if  its  truth  is  immediately  compelling, 
and  if  it  is  defended  in  a  single  step. 
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The  issue  oi  Socratic  hints  has  not  been  explicitly 
treated  in  the  context  of  judgment  under  uncertainty.  and 
there  are  no  rules  that  distinguish  fair  tests  of 
intuitions,  from  contrived  riddles  on  the  one  hand,  and  from 
Socratic  instruction  on  the  other.  Imagine.  for  example, 
hou  Socrates  might  have  taught  a  student  to  give  the  proper 
answer  to  the  following  question: 

"Which  hospital  --  a  large  or  a  small  one  —  will 
more  often  record  days  on  which  over  60%  of  the  babies 
born  were  boys?". 

This  is  a  difficult  question  for  Stanford  undergraduates 
(Kahneman  &  Tversky.  1972,  p.  441),  but  a  correct  answer  can 
be  elicited  in  a  series  of  easy  steps,  perhaps  as  follows: 

"Would  you  not  agree  that  the  babies  born  in  a 
particular  hospital  on  a  particular  day  can  be  viewed  as 
a  sample?" 

"Quite  right.  And  now,  would  you  have  the  same 
confidence  in  the  results  of  a  large  sample,  or  of  a 
small  one?" 

"Indeed.  And  would  you  not  agree  that  your 
confidence  is  greater  in  a  sample  that  is  less  likely  to 


be  in  error?" 
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"Of  course  you  had  always  known  that.  Mould  you  now 
tell  me  what  is  the  proportion  oi  boys  in  a  collection 
of  babies  uhich  you  consider  the  closest  to  an  ideal  of 
truth?" 

"Me  agree  again.  Does  that  not  meant  then#  that  a 
day  on  which  more  than  6 09$  of  babies  born  is  a  grave 
departure  from  that  ideal?" 

"And  so#  if  you  have  great  confidence  in  a  sample, 
should  you  not  expect  that  sample  to  reveal  truth  rather 
than  error?".  Etc. 

The  Socratic  procedure  is  a  heavy-handed  way  of  leading 
the  respondent  to  a  desired  response.  but  there  are  subtler 
ways  of  achieving  the  same  goal.  Fischhoff,  Slovic  and 
Lichtenstein  (1979)  showed  that  subjects  become  sensitive  to 
base  rates  and  to  the  reliability  of  evidence.  when  they 
encounter  successive  problems  that  vary  only  in  these 
critical  variables.  Although  these  investigators  did  not 
obtain  an  effect  of  sample  size  even  in  a  within-sub ject 
design.  such  effects  have  been  obtained  by  Evans  and  Dusoir 
(1977)  and  by  Bar-Hillel  (1979)  with  a  more  transparent 
formulation  and  more  extreme  sample  outcomes. 

The  hint  provided  by  parallel  problems  may  lead  subjects 
to  assign  weight  to  a  variable  that  is  actually  irrelevant 
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to  the  correct  response:  Fischhoff  and  Bar-Hillel  (1980) 
demonstrated  that  respondents  were  sensitive  to  irrelevant 
base-rate  information,  if  that  was  the  only  variable 
distinguishing  a  set  of  problems.  Indeed,  subjects  are 
prone  to  believe  that  any  feature  of  the  data  that  is 
systematically  varied  is  relevant  to  the  correct  response. 
Wi thin-subject  designs  are  associated  with  significant 
problems  of  interpretation  in  several  areas  of  psychological 
research  (Poulton,  1975).  In  studies  of  intuitions,  they 
are  liable  to  induce  the  effect  which  they  are  intended  to 
test . 

OH  THE  LIMITATIONS  OF  THE  QUEST I ON- ANSWERING  PARADIGM 

In  the  preceding  section  we  .J-ised  the  possibility  that 
within-sub ject  designs  and  Socratic  hints  could  prompt  the 
intuitions  under  study.  Th<S-  problem  is  actually  much 
broader.  Most  research  on  judgment  under  uncertainty  and  on 
inductive  inference  has  been  conducted  in  a  conversational 
paradigm  in  which  the  subject  is  exposed  to  information  and 
is  asked  to  answer  questions  or  to  estimate  values,  orally 
or  in  writing.  In  this  section  we  discuss  some  difficulties 
and  limitations  associated  with  this  question-answering 
paradigm. 

The  use  of  short  questionnaires  completed  by  casually 
motivated  subjects  is  often  criticized  on  the  grounds  that 
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subjects  would  act  differently  if  they  took  the  situation 
more  seriously.  However,  the  evidence  indicates  that  errors 
of  reasoning  and  choice  that  were  originally  established 
with  hypothetical  questions  are  not  eliminated  by  the 
introduction  oi  substantial  incentives  (Lichtenstein  £ 
Slovic .  1971.  1973;  Grether,  1979;  Grether  £  Plott.  1979; 
Tversky  £  Kahneman.  1981).  Hypothetical  questions  are 
appropriate  when  people  are  able  to  predict  how  they  would 
respond  in  a  more  realistic  setting.  and  when  they  have  no 
incentive  to  lie  about  their  responses.  That  is  not  to  say 
that  payoffs  and  incentives  do  not  affect  judgment.  Rather, 
we  maintain  that  errors  of  reasoning  and  choice  do  not 
disappear  in  the  presence  of  payoffs.  Heither  the  daily 
newspaper  nor  the  study  of  past  political  and  military 
decisions  support  the  optimistic  view  that  rationality 
pervails  when  the  stakes  are  high.  (Janis.  1972;  Janis  £ 
Mann.  1977;  Jervis,  1975). 

Perhaps  a  more  serious  concern  regarding  the  question- 
answering  paradigm  is  that  we  cannot  safely  assume  that 
"experimental  conversations"  in  which  subjects  receive 
messages  and  answer  questions  will  simulate  the  inferences 
that  people  make  in  their  normal  interaction  with  the 
environment.  Although  some  judgments  in  everyday  life  are 
made  in  response  to  explicit  questions,  many  are  not. 
Furthermore,  conversational  experiments  differ  in  many  ways 


from  normal  social  interaction. 


In  interpreting  the  subjects'  answers,  experimenters  ere 
tempted  to  assume  (i)  that  the  questions  merely  elicit  iron 
subjects  an  overt  expression  of  thoughts  that  would  have 
occurred  to  them  spontaneously,  and  (ii)  that  all  the 
information  given  to  the  subject  is  included  in  the 
experimental  message.  The  situation  is  quite  different  from 
the  subject's  point  of  vieu.  First,  the  question  that  the 
experimenter  asks  might  not  spontaneously  arise  in  the 
situation  that  the  experiment  is  meant  to  simulate.  Second, 
the  subject  is  normally  concerned  with  many  questions  that 
the  experimenter  never  thought  of  asking,  such  as:  "Is 
there  a  correct  answer  to  this  question?  Does  the 
experimenter  expect  me  to  find  it?  Is  an  obvious  answer  at 
all  likely  to  be  correct?  Does  the  question  provide  any 
hints  about  the  expected  answer?  Uhat  determined  the 
selection  of  the  information  that  I  was  given?  Is  some  of 
it  irrelevant  and  included  just  to  mislead,  or.  is  it  all 
relevant?"  The  single  overt  answer  that  the  experimenter 
observes  is  determined  in  part  by  the  subject's  answers  to 
this  cluster  of  tacit  questions.  And  the  experimental 
message  is  only  one  of  the  sources  of  information  that 
subjects  use  to  generate  both  the  covert  and  the  overt 
answers  (Orne,  1973). 

Following  Grice's  William  Jamas  lectures  in  1967  (Grice, 
197S),  a  large  body  of  literature  in  philosophy,  linguisites 
and  psycholinguistics  has  dealt  with  the  contribution  of  the 
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cooperat i vinos  principle  to  the  meaning  oi  utterances  ({or 
references,  see  Clark  £  Clark,  1977).  By  this  principle, 
the  listener  in  a  conversation  is  entitled  to  assume  that 
the  speaker  is  trying  to  be  "informative,  truthful,  relevant 
and  clear"  (Clark  £  Clark,  p.  560).  Grice  listed  several 
maxims  that  a  cooperative  speaker  uill  normally  follou.  For 
example,  the  maxim  of  quantity  prohibits  the  speaker  from 
saying  things  that  the  listener  already  knous,  or  could 
readily  infer  from  the  context  or  from  the  rest  of  the 
message.  It  is  by  this  maxim  that  the  statement  "John  tried 
to  clean  the  house"  conveys  that  the  attempt  uas 
unsuccessful:  the  listener  can  assume  that  a  successful 
attempt  uould  have  been  described  by  the  simpler  sentence: 
"John  cleaned  the  house". 

Subjects  come  to  the  experiment  with  lifelong  experience 
of  cooperativeness  in  conversation.  They  uill  generally 
expect  to  encounter  a  cooperative  experimenter,  although 
this  expectation  is  often  urong.  The  assumption  of 
cooperativeness  has  many  subtle  effects  on  the  subjects' 
interpretation  of  the  information  to  which  they  are  exposed. 
In  particular,  it  makes  it  exceptionally  difficult  for  the 
experimenter  to  study  the  effects  of  'irrelevant' 
information.  Because  the  presentation  of  irrelevant 
information  violates  rules  of  conversation,  subjects  are 
likely  to  seek  relevance  in  any  experimental  message.  For 
example,  Taylor  and  Crocker  (1979)  commented  on  the  fact 
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that  subjects'  impressions  of  a  person  are  affected  by 
statements  that  are  true  of  everybody,  e.g.»  "Mark  is  shy 
uith  his  professors".  But  the  subjects'  inference  that  Hark 
is  unusually  shy  could  be  justified  by  the  belief  that  a 
cooperative  experimenter  would  not  include  a  wholly 
redundant  statement  in  a  personality  description.  Similar 
issues  arise  in  other  studies  (e.g.,  Kahneman  C  Tversky, 
1973:  Nisbett,  Zukier  £  Lemley,  1981)  which  investigated  the 
impact  of  irrelevant  or  worthless  information. 

The  role  of  presuppositions  embedded  in  a  question  was 
illustrated  in  a  study  by  Loftus  and  Palmer  (1974),  who 
showed  that  eye-witnesses  give  a  higher  estimate  of  the 
speed  of  a  car  when  asked  "how  fast  was  the  car  going  when 
it  smashed  the  other  car?"  than  when  the  question  is  "how 
fast  was  the  car  going  when  it  hit  the  other  car?".  The  use 
of  the  wqrd  "smash"  in  the  question  implies  that  the 
questioner,  if  sincere  and  cooperative,  believes  that  the 
car  lias  going  fast. 

The  normative  analysis  of  such  an  inference  can  be 
divided  into  two  separate  problems.  (i)  should  the  witness 
be  affected  by  the  question  in  forming  a  private  opinion  of 
the  speed  of  the  car?  (ii)  Should  the  witness  be  affected 
by  the  question  in  formulating  a  public  estimate?  The 
answer  to  (i)  must  be  positive  if  the  question  conveys  new 
information.  The  answer  to  (ii)  is  less  clear.  On  the  one 
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hand.  it  appears  inappropriate  for  the  reply  to  a  question 
to  echo  information  contained  in  the  question.  On  the  other 
hand.  the  cooperative  witness  is  expected  to  give  the  best 
possible  estimate  in  responding  to  a  question  about  a 
quantity.  Uhat  is  the  witness  to  do  if  that  estimate  has 
just  been  influenced  by  the  question?  Should  the  reply  be: 
"before  you  asked  me.  1  would  have  thought  .  .  ."?  Whatever 
the  normative  merits  of  the  case.  the  evidence  indicates 
that  people  are  often  unable  to  isolate  past  opinions  from 
current  ones.  or  to  estimate  the  weight  of  factors  that 
affected  their  views  (Fischhoff.  1977;  Goethals  £  Reckman. 
1973;  Nisbett  £  Wilson.  1977;  Ross  £  Lepper.  1980). 

Our  research  on  anchoring  (Tversky  £  Kahneman,  1974) 
further  illustrates  the  potency  of  subtle  suggestions.  In 
one  study  we  asked  a  group  of  subjects  to  assess  the 
probability  that  the  population  of  Turkey  was  greater  than  5 
million.  and  another  group  to  assess  the  probability  that 
the  population  of  Turkey  was  less  than  65  million. 
Following  this  task.  the  two  groups  recorded  their  best 
guesses  about  the  population  of  Turkey;  the  median  estimates 
were  17  million  and  35  million,  respectively,  for  the  groups 
exposed  to  the  low  and  to  the  high  anchors.  These  answers 
can  also  be  rationalized  by  the  assumption  that  the  values 
which  appear  in  the  probability  questions  are  not  very  far 
from  the  correct  one. 


Ue  have  argued  that  suggestion  effects  can  sometimes  be 
justified  because  there  is  no  clear  demarcation  between 
suggestion  and  information.  It  is  important  to  note, 
however,  that  people  do  not  accept  suggestions  because  it  is 
appropriate  to  do  so.  In  the  first  place.  they  usually  do 
not  know  that  they  have  been  affected  by  a  suggestion 
(Loftus.  1979;  Hisbett  £  Wilson.  1977).  Second.  similar 
suggestion  effects  are  observed  even  when  respondents  cannot 
reasonably  believe  that  an  anchor  which  they  are  given 
conveys  information.  Subjects  who  were  required  to  produce 
estimates  of  quantities  by  adjusting  up  or  down  from  a 
randomly  generated  value  showed  strong  evidence  of  anchoring 
effects  (Tversky  £  Kahneman.  1974).  It  is  not 
suggestibility  as  such  that  is  troublesome,  but  the  apparent 
inability  to  discard  uninformative  messages. 

When  subjects  are  required  to  indicate  their  response  by 
choosing  an  answer  from  a  list.  or  by  constructing  a 
probability  distribution  over  a  given  set  of  alternatives, 
the  experimenter's  choice  of  categories  could  be 
informative.  Loftus  (1979)  has  shown  that  respondents 
report  many  more  headaches  per  week  when  the  response  scale 
is  expressed  as  1-5.  5-10.  10-15,  etc.,  than  when  the  scale 
is  expressed  as  1-3,  3-5.  5-7,  etc.  In  this  case,  the  scale 
could  legitimately  affect  the  boundaries  of  what  is  to  be 
called  a  headache.  Even  when  such  reinterpretations  are  not 
possible.  subjects  may  be  expected  to  favor  the  middle  of 
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the  range  in  their  estimates  of  quantities,  and  to  produce 
subjective  probability  distributions  in  which  each  category 
is  assigned  a  non-neg 1 ig i bl e  probability  (Olson,  1976; 
Parducci  ,  1965) . 

Suggestions  implied  by  the  questionnaire  could  also 
contribute  to  a  result  observed  by  Pischhoff,  Slovic  and 
Lichtenstein  (1978)  who  asked  naive  subjects  and  experienced 
garage  mechanics  to  evaluate  the  probability  of  different 
malfunctions  that  could  cause  failure  in  starting  a  car. 
They  found  that  the  estimated  probability  of  the  category 
"all  other  problems"  was  quite  insensitive  to  the 
completeness  of  the  list,  and  was  hardly  increased  when  a 
major  factor  (e.g.,  the  entire  electrical  system)  was 
deleted  from  that  list. 

Even  subtle  and  indirect  clues  can  be  effective.  In  a 
recent  study  ue  gave  subjects  the  following  information: 
"Mr.  A  is  Caucasian,  age  33.  He  weighs  190  pounds".  One 
group  of  subjects  were  asked  to  guess  his  height..  Other 
subjects  also  guessed  his  height,  after  first  guessing  his 
waist  size.  The  average  estimate  was  significantly  higher 
in  the  first  group,  by  about  one  inch.  Ue  surmise  that 
subjects  who  first  guessed  waist  size  attributed  more  of  Hr. 
A's  weight  to  his  girth  than  did  subjects  who  only  guessed 
his  height. 
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He  conclude  that  the  conversational  aspect  of  judgment 
studies  deserves  more  careful  consideration  than  it  has 
received  in  past  research.  our  own  included.  He  cannot 
always  assume  that  people  will  or  should  make  the  same 
inferences  from  observing  a  fact  and  from  being  told  the 
same  fact.  because  the  conversational  rules  that  regulate 
communication  between  people  do  not  apply  to  the  information 
that  is  obtained  by  observing  nature.  It  is  often  difficult 
to  ask  questions  without  giving  (useful  or  misleading)  clues 
regarding  the  correct  answer.  and  without  conveying 
information  about  the  expected  response.  A  discussion  of  a 
related  normative  issue  concerning  the  interpretation  of 
evidence  is  included  in  Bar-Killel  and  Falk  (1980). 

Naturally.  the  biasing  factors  that  we  have  mentioned 
are  likely  to  have  most  impact  in  situations  of  high 
uncertainty.  Subjects'  interpretations  of  the 
experimenter's  conversational  attitude  will  not  be  given 
much  weight  if  they  conflict  with  confident  knowledge  of  the 
correct  answer  to  a  question.  In  the  gray  area  where  most 
judgment  research  is  carried  out.  however.  variations  of 
conversational  context  can  affect  the  reasoning  process  as 
well  as  the  observed  response. 


JUDGMENTAL  ERRORS:  POSITIVE  AND  NEGATIVE  ANALYSES 
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It  is  oiten  useful  to  distinguish  between  positive  and 
negative  accounts  of  judgmental  errors.  A  positive  analysis 
focuses  on  the  factors  that  produced  a  particular  incorrect 
response;  a  negative  analysis  explains  why  the  correct 
response  was  not  made.  For  example*  the  positive  analysis 
of  a  child's  failure  in  a  Piagetian  conservation  task 
attempts  to  specify  the  factors  that  determine  the  child's 
response*  e.g.»  the  relative  height  or  surface  area  of  the 
tuo  containers.  A  negative  analysis  of  the  same  behavior 
would  focus  on  the  obstacles  that  make  it  difficult  for  the 
child  to  acquire  and  to  understand  the  conservation  of 
volume.  In  the  investigation  of  judgment  under  uncertainty, 
positive  analyses  are  concerned  with  the  heuristics  that 
people  use  to  make  judgments*  estimates  and  predictions. 
Negative  analyses  are  concerned  with  the  difficulties  of 
understanding  and  applying  elementary  rules  of  reasoning. 
In  the  case  of  an  error  of  comprehension*  the  negative 
analysis  focuses  on  the  obstacles  that  prevent  people  from 
discovering  the  relevant  rule  on  their  own,  or  from 
accepting  simple  explanations  of  it.  The  negative  analysis 
of  an  error  of  application  seeks  to  identify  the  ways  in 
which  the  coding  of  problems  may  mask  the  relevance  of  a 
rule  that  is  known  and  accepted. 

In  general*  a  postive  anaysis  of  an  error  is  most  useful 
when  the  same  heuristic  explains  judgments  in  a  varied  set 
of  problems*  where  different  normative  rules  are  violated. 
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Correspondingly,  a  negative  analysis  is  most  illuminating 
uhen  people  consistently  violate  a  rule  in  different 
problems,  but  make  errors  that  cannot  be  attributed  to  a 
single  heuristic.  It  then  becomes  appropriate  to  ask  uhy 
people  failed  to  learn  the  rule,  if  routine  observations  of 
everyday  events  offer  sufficient  opportunities  for  such 
learning.  It  also  becomes  appropriate  to  ask  uhy  people 
resist  the  rule,  it  they  are  not  convinced  by  simple  valid 
arguments.  The  difficulties  of  learning  statistical 
principles  from  everyday  experience  have  been  discussed  by 
several  authors,  notably  Goldberg  (1968),  Einhorn  and 
Hogarth  (1978),  and  Nisbett  and  Ross  (1980).  Failures  of 
learning  are  commonly  traced  to  the  inaccessibility  of  the 
necessary  coding  of  relevant  instances,  or  to  the  absence  of 
corrective  feedback  for  erroneous  judgments.  The  resistance 
to  the  acceptance  of  a  rule  is  normally  attributed  to  its 
counter-intuitive  nature.  As  an  example,  ue  turn  nou  to  the 
analysis  of  the  reasons  for  the  resistance  to  the  principle 
of  regressive  prediction. 

Studies  of  intuitive  prediction  have  provided  much 
evidence  for  the  prevalence  of  the  tendency  to  make 
predictions  that  are  radical,  or  insufficiently  regressive. 
(For  a  recent  review  of  this  literature  see  Jennings, 
Amabile  &  Ross,  1982;)  In  earlier  articles  ue  offered  a 


positive  analysis  of  this  effect  as  a  manifestation  of  the 
representativeness  heuristic  (Kahneman  £  Tversky,  1973, 
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1979).  However,  as  ue  shall  see  below,  there  are  reasons  to 
turn  to  a  negative  analysis  in  order  to  provide  a  more 
comprehensive  treatment. 

A  negative  analysis  is  of  special  interest  tor  errors  oi 
comprehension,  in  which  people  find  the  correct  rule  non- 
intuitive.  or  even  counter-intuitive.  As  most  teachers  of 
elementary  statistics  will  attest,  students  find  the  concept 
of  regression  very  difficult  to  understand  and  apply  despite 
a  lifetime  of  experience  in  which  extreme  predictions  were 
most  often  too  extreme.  Sportcasters  and  teachers.  for 
example.  are  familiar  with  manifestations  of  regression  to 
mediocracy:  exceptional  achievements  are  followed  more 
often  than  not  by  disappointment.  and  failures  by 
improvement . 

Futhermore,  when  the  regression  of  a  criterion  variable 
on  a  predictor  is  actually  linear.  and  when  the  conditional 
distributions  of  the  criterion  (for  fixed  values  of  the 
predictor)  are  symmetric,  the  rule  of  regressive  prediction 
can  be  defended  by  a  compelling  argument:  it  is  sensible  to 
make  the  same  prediction  for  all  cases  that  share  the  same 
value  of  the  predictor  variable,  and  it  is  sensible  to 
choose  that  prediction  so  that  the  mean  and  the  median  of 
the  criterion  value,  for  all  cases  that  share  the  same 
predicted  value  Y.  will  be  equal  to  Y.  This  rule,  however, 
conflicts  with  other  intuitions,  some  of  which  are  discussed 
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belou. 

(i)  "An  optimal  rule  of  prediction  should  at  least 
permit,  if  not  guarantee,  perfectly  accurate  predictions  for 
the  entire  ensemble  of  cases."  The  principle  of  regressive 
prediction  violates  this  seemingly  reasonable  requirement. 
It  yields  a  set  of  predicted  values  which  has  less  variance 
than  the  corresponding  set  of  actual  criterion  values,  and 
thereby  excludes  the  possibility  of  a  set  of  precisely 
accurate  predictions.  Indeed,  the  regression  rule 
guarantees  that  an  error  will  be  made  on  each  pair  of 
correlated  observations:  we  can  never  find  a  son  whose 
height  was  correctly  predicted  from  his  father's  height,  and 
whose  height  also  allowed  an  accurate  prediction  of  the 
father's  height,  except  when  both  values  are  at  the  mean  of 
the  height  distribution.  It  appears  odd  that  a  prediction 
rule  that  guarantees  error  should  turn  out  to  be  optimal. 

(ii)  "The  relation  between  an  observation  and  a 
prediction  based  on  it  should  be  symmetric."  It  seems 
reasonable  to  expect  that,  if  B  is  predicted  from  knowledge 
of  A,  then  A  should  be  the  appropriate  prediction  when  B  is 
known.  Regressive  preditions  violate  this  symmetry,  of 
course,  since  the  predictions  of  the  two  variables  from  each 
other  are  not  governed  by  the  same  regression  equation.  A 


related  asymmetry  is  encountered  in  comparing  regressive 
predictions  to  the  actual  values  of  the  criterion  variable. 
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Regressive  predictions  are  unbiased.  in  the  sense  that  the 
mean  criterion  value.  over  all  cases  ior  which  a  particular 
value  Y  was  predicted,  is  expected  to  be  Y.  However,  if  we 
consider  all  the  cases  ior  which  the  criterion  value  was  Y, 
it  will  be  found  that  the  mean  of  their  predicted  scores 
lies  between  Y  and  the  group  average.  These  asymmetries  are 
puzzling  and  counter-intuitive  ior  intelligent  but 
statistically  naive  persons. 

The  asymmetries  of  regressive  prediction  are  especially 
troubling  when  the  initial  observation  and  the  criterion  are 
generated  by  the  same  process  and  are  not  distinguishable  a 
priori.  as  in  the  case  oi  repeated  sampling  from  the  same 
population.  or  in  the  case  oi  parallel  forms  oi  the  same 
test.  The  only  mode  oi  prediction  that  satisfies  symmetry 
in  such  situations  is  an  identity  rule,  where  the  score  on 
the  second  form  is  predicted  to  be  the  same  as  the  initial 
observation.  The  principle  of  regressive  prediction 
introduces  a  distinction  for  which  there  is  no  obvious 
reason:  how  is  it  possible  to  predict  the  sign  of  the 
difference  between  two  values  drawn  from  the  same 
population,  as  soon  as  one  of  these  values  is  known? 

(iii)  "Any  systematic  effect  must  have  a  cause."  The 
difference  between  initial  observations  and  the 
corresponding  criterion  values  is  a  fact,  which  can  be 
observed  in  any  scatter  plot.  However,  it  appears  to  be  an 
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effect  without  a  cause.  In  a  test-retest  situation.  for 
example,  the  knowledge  that  the  first  score  uas  high 
entails  the  prediction  that  the  second  will  be  lower,  but 
the  first  observation  does  not  casue  the  second  to  be  low. 
The  appearance  of  an  uncaused  effect  violates  a  powerful 
intuition.  Indeed,  the  understanding  of  regression  is 
severely  hindered  by  the  fact  that  any  instance  of 
regression  on  which  one  stumbles  by  accident  is  likely  to  be 
given  a  causal  explanation.  In  the  context  of  skilled 
performance,  for  example,  regression  from  an  initial  test  to 
a  subsequent  one  is  commonly  attributed  to  intense  striving 
after  an  initial  failure  and  to  overconfidence  following  an 
initial  success.  It  is  often  difficult  to  realize  that 
performers  uould  regress  even  without  knowledge  of  results, 
merely  because  of  irreducible  unreliability  in  their 
performance.  The  regression  of  the  first  performance  on  the 
second  is  also  surprising  because  it  cannot  be  given  a 
simple  causal  explanation. 

Ue  have  sketched  above  a  negative  analysis  of  people's 
difficulties  to  understand  and  apply  the  concept  of 
regressive  prediction.  Me  propose  that  people  have  strong 
intuitions  about  statistical  prediction,  and  that  some 
normatively  correct  principles  are  counter-intuitive 
precisely  because  they  violate  existing  intuitions.  In  this 
view,  the  "principles"  that  people  adopt  represent 
significant  beliefs,  not  mere  rationalizations,  and  they 
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play  a  substantial  role  in  retarding  the  learning  of  the 
correct  rules.  These  beliefs,  however,  are  often 
contradictory  and  hence  unrealizable.  For  example.  it  is 
impossible  to  construct  a  non-degenerate  joint  distribution 
of  the  height  of  fathers  and  (first)  sons  so  that  the  mean 
height  of  a  father  will  be  an  unbiased  predictor  of  the 
height  of  his  son  and  the  height  of  a  son  will  be  an 
unbiased  predictor  of  the  height  of  his  father. 

In  conclusion.  ue  have  proposed  that  some  errors  and 
biases  in  judgment  under  uncertainty  call  for  a  dual 
analysis:  a  positive  account  that  explains  the  choice  of  a 
particular  erroneous  response  in  terms  of  heuristics.  and  a 
negative  account  that  explains  why  the  correct  rule  has  not 
been  learned.  Although  the  two  analyses  are  not 
incompatible,  they  tend  to  highlight  different  aspects  of 
the  phenomenon  under  study.  The  attempt  to  integrate  the 
positive  and  the  negative  accounts  is  likely  to  enrich  the 
theoretical  analysis  of  inductive  reasoning. 

Summary 

Ue  addressed  in  this  essay  three  clusters  of 


methodological  and  conceptual  problems  in  the  domain  of 
judgment  under  uncertainty.  First,  ue  distinguished  between 
errors  of  application  and  errors  of  comprehension  and 


discussed  different  methods  for  studying  statistical 
intuitions.  Second.  we  revieued  some  limitations  of  the 
question-answering  paradigm  of  judgment  research  and 
explored  the  effects  of  tacit  suggestions.  Socratic  hints 
and  rules  of  conversation.  Third,  ue  discussed  the  roles  of 
positive  and  negative  explanations  of  judgmental  errors. 

The  considerations  raised  in  this  paper  complicate  the 
empirical  and  the  theoretical  analysis  of  judgment  under 
uncertainty;  they  also  suggest  new  directions  for  future 
research.  Ue  hope  that  a  deeper  appreciation  of  the 
conceptual  and  the  methodological  problems  associated  with 
the  study  of  statistical  intuitions  will  lead  to  a  better 
understanding  of  the  complexities.  the  subtleties.  and  the 
limitations  of  human  inductive  reasoning. 
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